Performance characterization of containerization for HPC workloads on InfiniBand clusters: an empirical study

Abstract Containerization technology offers an appealing alternative for encapsulating and operating applications (and all their dependencies) without being constrained by the performance penalties of using Virtual Machines and, as a result, has got interest High-Performance Computing (HPC) community to obtain fast, customized, portable, flexible, reproducible deployments workloads. Previous work on this area demonstrated that containerized HPC can exploit InfiniBand networks, but ignored potential multi-container which partition processes belong each application into multiple containers in host. Partitioning be useful when virtual machines constraining them single NUMA (Non-Uniform Memory Access) domain. This paper conducts systematical study with different network fabrics protocols, focusing especially Infiniband networks. We analyze impact container granularity its processor memory affinity improve applications’ performance. Our results show default Singularity achieve near bare-metal does not support fine-grain deployments. Docker Singularity-instance have similar behavior terms deployment schemes affinity. differs several depends well communication patterns message size. Moreover, are also more impacted computation allocation, because that, they better.